Visual Sorting Method Based on Multi-Modal Information Fusion

Visual sorting of stacked parcels is a key issue in intelligent logistics systems. In order to improve the success rate express and effectively obtain parcels, visual method based on multi-modal information fusion (VS-MF) proposed this paper. Firstly, an object detection network (OD-MF) proposed. The global gradient feature extracted from depth as self-attention module. More spatial features are learned by network, accuracy improved significantly. Secondly, segmentation Swin Transformer (MS-ST) detect optimal positions poses parcels. fine-grained relationships between them gained adding models. Frequency domain used supervision signals pickable areas infer occlusion degrees A strategy for also ensure stability system. Finally, system with 6-DOF robot constructed complete task verified experiments.